Watermarking of video signals is, generally, the inclusion within the video itself of additional information. This can be useful to provide an embedded identification of the source of a video, to keep track of where and for how long a video is played, and to communicate information via the video to an ancillary device. Prior art techniques for watermarking video signals typically encoded the additional information in an analog format within the video itself using the luminance of the video to carry the additional information. However, the human visual system is very sensitive to the luminance signal, and so a person viewing a watermarked signal easily perceives distortion which is caused by the changes made to the video signal to convey the additional information when there is an attempt to increase the bit rate of the additional information beyond a certain point, e.g., beyond 120 bits per second. Thus, although the prior art's techniques of watermarking of video signals has had some success in certain applications, such success has been limited by the extremely small bit rate that is achievable without perceivable distortion by a person viewing the video signal carrying the additional information.
In previously filed U.S. patent application Ser. No. 10/342,704, which is incorporated by reference as if set forth fully herein, I, along with my coinventor, recognized that the human visual system is much less sensitive to chrominance than to luminance. Therefore, we developed a system for digital watermarking a video signal that inserts the additional information of the watermarking signal on the chrominance component of the video signal rather than on its luminance signal. Thus, the additional information is “impressed” upon the chrominance component of the video signal. Advantageously, although there may be significant distortion of the chrominance component, especially when the additional information has higher bit rates than is achievable without perceivable distortion by the prior art, nevertheless such distortion will not be detected by the human visual system, provided it is appropriately managed. Thus, the additional information can have a higher bit rate as compared with that achievable by the prior art, e.g., bit rates greater than 150 bits per second can be achieved. Further advantageously, the additional data can be recovered from the video signal even after the video signal watermarked with the additional data is compressed using the Motion Picture Expert Group (MPEG)-1 and MPEG-2 encoding systems.
I have recognized that the system of U.S. application Ser. No. 10/342,704 required multiple frames of the video to accurately transmit the information on the chrominance portion. This can be disadvantageous in some applications, e.g., where finer granularity of the watermarking is required so as to provide better response time and improved resistance to temporal tampering with the video, and under certain conditions, e.g., when there is a scene change, which can occur rapidly in the case of high speed movement such as a chase scene.
In concurrently filed U.S. application Ser. No. 10/673,892 the watermarking of video is improved by having one or more bits of watermark data carried via an average value of the chrominance component of each of various blocks of the video signal, on up to a per-frame basis. More specifically, one or more bits of the watermark data may be effectively placed into specified bit positions of the average value of at least a selected portion of the chrominance component of up to each block of up to each frame. Note that typically, there are two chrominance portions, e.g., U and V when the video is represented in the YUV format, in the chrominance component. More specifically, each block of each frame of the original video signal may be modified to carry its own independent one or more bits of the watermark data in the average value of a chrominance portion that is selected to carry the watermark data for that block. Conceptually then, the value of the bit position of the average value of the selected chrominance portion that is to contain the watermark data for a block may be thought of as being replaced by the value of the bit of watermark data to be carried in that block.
Only one of the chrominance portions carries watermark data for any particular block. The chrominance portion selected to carry the watermark data for any block may be independently selected for that block. The bit position of the average value that is replaced is one of the bits of the integer portion of the average value.
If necessary, the values of the selected chrominance portion of individual pixels of a block may be adjusted in order to cause the bit of the average value of the selected chrominance portion of the block that is to carry the watermark data to be the same as the value of the watermark data bit. This may be achieved by changing the value of the selected chrominance portion of various pixels in the block such that over the entire block the average of the value of the chrominance portion is changed so that the value of the select bit position of the average conforms to the value of the watermark data that is being placed in the selected bit position.
If the value of the bit of the average value to contain the watermark data is already the same as the value of the watermark data bit, no change may be performed to any of the pixels of the block. However, if the value of the bit of the average value to contain the watermark data is the complement of the value of the watermark data bit to be carried by the block, at least the minimum change to the average value that will cause the bit to be changed to the value of the watermark data bit is performed on the average value. For example, if the bit of the average value to contain the watermark data is the second least significant bit of the integer portion of the average value, such a bit may always be changed to its complementary value by either adding or subtracting one to the average value. Doing so is preferable to adding two, which may also be used to always change the bit to its complementary value, because it introduces less change in the value of the average, and hence less change in the block, thereby reducing the chance of introducing a viewer-perceivable artifact. The change to the average value of the selected chrominance portion of a block is implemented by adding or subtracting—, which may be accomplished by adding negative numbers—, a value to the selected chrominance portion of ones of the pixels of the block until the desired change in the average thereof is achieved.
When using block-based frequency domain encoding of the video, such as one of the motion picture experts group (MPEG) standards, e.g., MPEG-1, MPEG-2, MPEG-4, the substitution of the bit of watermarked data may be achieved by adjusting the value of the DC coefficient, which corresponds to the average value, of at least one of the chrominance matrices for the block. For example, the second least significant bit of the DC coefficient for a block is replaced with the value of the watermark bit that is desired to be impressed on the block.
Which bit of the average value of the chrominance portion is designated to carry the watermark data may be a function of a texture variance of the block. It is advantageous to increase the significance of the bit position carrying the watermark data as the texture variance increases, because MPEG-like coding employs greater quantization step sizes for higher texture variances, and the use of such greater quantization step sizes could result in the elimination, e.g., filtering out, of the watermarking data if it is not positioned significantly enough. When using more significant bit positions, the values to be added or subtracted from the average value in order to change the value of a bit position carrying the watermark data to its complementary value may be greater than one. Any texture variance may be used, e.g., the texture variance of Y, U, or V, or a combination thereof.
Whether or not the bit position carrying the watermark data was changed to its complement, a “margin” value may be added to the average value in order to better ensure that the bit of watermark data carried by the average value of the block survives any MPEG-like encoding, while minimizing the likelihood of perceivable artifacts resulting.
A receiver determines which of the chrominance components of a block are carrying the watermark data and extracts the bit of watermark data from the selected bit position of the integer portion of the average value of that chrominance component. The selected bit position may be determined from a texture variance of the block, e.g., the texture variance of the determined chrominance portion of the block or the texture variance of the luminance component.
Advantageously, better response time and improved resistance to temporal tampering with the video is achieved with respect to the prior art. Further advantageously, scene changes do not introduce errors into the impressed data. Yet an additional advantage is that even if the original pixel domain version of the video is not available, but only a block-based frequency domain encoded version thereof, the video may be watermarked without conversion back to the pixel domain.